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Letter from the Editor 


Summer brings time to reflect and recharge. The Summer 2017 volume of AIR Professional Files 
presents four articles with intriguing ideas to consider as you plan for the next academic year. | 


Data governance is a pressing issue for many IR professionals, as sources of data proliferate and os 
Association for 


challenge our ability to control data integrity. In her article, Institutional Data Quality and the Institutional Research 
Data Integrity Team, McGuire synthesizes and interprets results from 172 respondents to an AIR- 
administered survey of postsecondary institutions on their data integrity efforts. She describes the current state of data governance 


and offers strategies to encourage institutional leaders to invest in data quality. 


Those of us who work in assessment often take it for granted that assessment results will be used for learning improvement. Fulcher, 
Smith, Sanchez, and Sanders challenge this assumption by analyzing information from program assessment reports at their own 
institution. Needle in a Haystack: Finding Learning Improvement in Assessment Reports uncovers many possible reasons for the gap 
between obtaining evidence of student learning and using that evidence for improvement. The authors suggest ways to promote 
learning improvement initiatives, and share a handy rubric for evaluating assessment progress. 


Institutional researchers are beset with requests to form peer groups, and it seems that no one is ever satisfied with the results. 

Two articles in this volume present very different methodologies for forming sets of comparison institutions. In her article, A Case 
Study to Examine Three Peer Grouping Methodologies, D‘Allegro compares peer sets generated by different selection indices. She 
offers guidance for applying each index and encourages cautious interpretation of results. Rather than rummaging around for 

the perfect peer set, Chatman proposes creating a clone, or doppelganger university, one that is constructed from disaggregated 
components drawn from diverse data sources. In Constructing a Peer Institution: A New Peer Methodology, he walks us through the 
process of creating peers for faculty salaries, instructional costs, and faculty productivity. While the constructed peer approach has its 
challenges, the appeal of achieving a perfect fit peer is undeniable. 


| hope your summer “reflection” inspires you to share your work with your IR colleagues through AIR Professional Files. 


Sincerely, 


Sharron L. Ronco 
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Abstract 

This study considered three selection 
indices to choose institutional peers: 
(a) proximity, (b) percentile, and (c) 
normative. Although conceptually 
similar, only the proximity selection 
index had been previously studied. 
The purpose of this paper is threefold. 
First, the procedures used to generate 


the peer sets for each selection index 
are provided. Second, an empirical 
investigation was conducted to 
compare the institutional peers 
chosen by each selection index using 
those procedures. Third, the stability 
of peer selection over time was also 
ascertained from that enquiry. 


Compiled separately from two data 
sets extracted three years apart, 

the three selection indices under 
investigation yielded remarkably 
different sets of peers. Fewer than half 
of the institutions used in this study 
were identified as peers at both points 
of time. Additional analyses revealed 
that the underlying distributions of 
the characteristics used to select peers 
might be just as influential as the 
characteristics themselves. The results 
did not produce sufficient evidence 

to endorse any one of the selection 
indices, but instead suggest that a 
combination of selection indices might 
be superior to any one selection index 
alone. 


BACKGROUND 


The continued increase in public 
scrutiny of higher education, the 
expanded demands of accountability, 
and the overall cynicism of the value 
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of higher education have put colleges 
and universities on high alert. To 
counter this skepticism, colleges and 
schools have increased their efforts 

to evaluate their quality, efficiency, 
and effectiveness (Ruben, 2004). A 
growing and important segment of 
that evaluation is the comparison 

and benchmarking to like institutions 
(Qayoumi, 2012). Therefore, peer 
selection has become more prevalent. 
Moreover, higher education has seen 
the benefit of using peer comparisons 
and benchmarking to inform decision 
making and strategic planning. 


This research builds on previous work 
that examined the methodology to 
choose a set of institutional peers. 
Specifically, that research investigated 
the usefulness of the proximity 
selection index and proposed 
standardized equation to foster 

ease of replication. In that work, the 
proximity selection index was deemed 
to be an appropriate methodology 

for the selection of a generic set of 
institutional peers (D’Allegro & Zhou, 
2013). For this research, an institutional 
peer is defined as institutions that 

are similar with regard to certain 
delineating factors (Anderes, 1999; 
Trainer, 2008). A selection index is 

a numerical designation system 
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to indicate the extent to which an 
institution is a potential peer. 


Faculty proposed to the researcher two 
different approaches to peer selection 
indices that were not considered in 

the researcher's previously published 
work. The faculty's suggestions seemed 
rational because their methodologies 
might temper potential irregularities 

in the data. Particularly, their proposed 
selection indices either (a) relied on 
statistics that were less susceptible to 
the perils of non-normal distributions 
than the standard deviation used 

in the proximity selection index or 

(b) standardized the distribution so 
that imperfections in the data were 
minimized. As will be discussed in the 
“Methodology” and “Results” sections, 
non-normal distributions can acutely 
affect the set of peer institutions that 
are selected. This further confirmed 
that the process for determining 

peers seems to be arbitrary (Anderes, 
1999). Accordingly, there is little or no 
evidence to the quality or adeptness of 
many processes to select a set of peers. 
Careful planning and investigation 

of the criteria used to select a set of 
institutional peers is still advised, but 
the researcher realized the frailty of 
even the most careful undertaking of 
selecting a set of institutional peers, 
including the conclusions of previous 
research. 


At the heart of the paper is the 
description of three different selection 
indices and the ensuing peer sets 
created by each. Those selection 
indices were similar to the nearest 
neighbor rationale (McLaughlin, 
Howard, & McLaughlin, 2011). For all 
three selection indices, the distance 
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between any given institution or 
comparison institution and the 

target institution on predetermined 
parameters was calculated. The 
divergence among selection indices 

is their underlying distributions. 
Correspondingly, the primary purposes 
of the study were to: (a) determine and 
document the differences, if any, in 

the institutional peer sets produced by 
each selection index; (b) conclude, from 
any differences, what index is best; 

and (c) ascertain the stability of peer 
selection over time. 


METHODOLOGY 


This study does not abandon previously 
applied principles and, as such, uses 

a variety of sources and methods to 
maintain a practical balance between 
stakeholder judgment and statistical 
analysis (Trainer, 2008). Credibility of 
the institutional peer sets relies on 
constituent input. Not only were faculty 
and staff consulted for this compilation, 
but in addition the concept for the 
alternative selection indices arose 

from the propositioned reasoning of 
two faculty members. Hence, selection 
methodologies were based primarily 
on constituent suggestions and on 
other documented peer selections. 


In the original research, an attempt 
to find a quick, pragmatic method 

to choose a set of peers from two 

or three institutional characteristics 
was unsuccessful. Using different 
combinations of those institutional 
characteristics, it was discovered that 
the resulting peer sets were similar to 
the target institution with respect to 
some data elements but different with 
respect to others. Those differences 


were substantial enough to render 
the selection process ineffectual. 

This reinforces previous findings that 
institutional characteristics alone are 
not sufficient in choosing institutional 
peers (Shin, 2009). 


Instead, a more-informed and more- 
comprehensive process was tested. 
The selection process entailed five 
steps outlined by D’Allegro and Zhou 
(2013): (a) identifying an initial set of 
peers, (b) choosing the preliminary 
set of variables, (c) transforming 

and standardizing variables, (d) 
determining the best set of variables 
to use, and (e) establishing the best 
selection strategy. This research is 
fundamentally undistinguishable 
from that research except for the last 
step. Therefore, a pithy summary of 
Steps 1-4 are provided, along with a 
comprehensive description of Step 5. 


1. Identifying an Initial Set of 
Peers 

The initial set of peers was selected a 
priori to this study. To recap, an initial 
set of institutional characteristics was 
identified to eliminate from further 
analysis institutions that would not 
realistically be considered a peer 

of the target institution. The initial 
set of institutions was chosen from 
an original list of private, nonprofit 
institutions that submitted data to the 
Integrated Postsecondary Education 
Data System (IPEDS) from the Data 
Compare Institutions website. The list 
was generated using the EZ group 
option (National Center for Education 
Statistics [NCES], 2012). Data for 
these institutions were collected for 
2010 and 2011; these were the most 
recent data available at the time of the 


previous study. An updated data set 
was identically assembled using 2014 
and 2015 information; these were the 
most recent data available at the time 
of this study. Note that for the target 
institution, the 2015 Basic Carnegie 
Classification did not change from 
2010 (Carnegie Foundation, 2015). 
Furthermore, only the basic 2015 Basic 
Carnegie Classification was currently 
available on the EZ group option. Lists 
for both time periods were generated 
using the following criteria: (a) private 
not-for-profit institutions, 4-year or 
above; (b) highest degree awarded 
either a bachelor’s degree, master’s 
degree, or both; (c) baccalaureate 
college for arts and sciences, or 
baccalaureate college balanced arts 
and sciences, diverse fields; (d) enrolled 
full-time undergraduate students; (e) 
institution size between 1,000 and 
9,999 students; (f) Title IV participant 
(federal financial aid eligibility); 

(h) located in the United States or 
designated as a U.S. Service School 
(e.g., U.S. Naval Academy), and (i) not a 
tribal college. These parameters align 
with the characteristics of the target 
institution. This is also on par with 
selection parameters recommended 
by previous studies (Anderes, 1999). 
As a result of applying these criteria, 
285 institutions were selected for the 
previous study while the updated listed 
yielded 232 institutions. 


2. Choosing the Preliminary Set 
of Variables 

Other pertinent information was 
collected for each of these institutions. 
Relevance in the context of selecting 
peers are those data points that 
indicate the institution's priorities 
(Anderes, 1999; Cohodes & Goodman, 


2012). For the most part, an institution’s 
focus is on quality. As such, the target 
institution's own Key Performance 
Indicators (KPls) were the starting 
point. KPls are a mix of approximately 
20 output or direct measures of quality 
and input or influencers of quality. 
Therefore, the initial set of variables 
chosen either had some influence on 
quality or included direct measures 

of institutional performance. Faculty 
and staff were also asked to rate the 
importance of each KPI, being mindful 
of the importance of using both input 
and output variables in the peer 
selection process. 


The data also had to be easy to access 
for all or most institutions. Several 
sources were considered including: 

(a) National Survey of Student 
Engagement (NSSE) benchmarks, 

(b) American Association of 

University Professors (AAUP) Faculty 
Compensation Survey (2012), (c) Noel 
Levitz Student Satisfaction Inventory 
(NLSSI), and (d) U.S. News & World 
Report rankings (U.S. News & World 
Report, 2015). Nevertheless, not all 
institutions participate in the NSSE 

or NLSSI or administer these surveys 
within a reasonable time period to 
avail comparisons. Also, detailed AAUP 
faculty salary data are not available for 
many institutions. Consequently, data 
were obtained from IPEDS or the U.S. 
News & World Report rankings. 


The preliminary set of 28 variables 
are shown in Appendix A, along 
with the institutional characteristics 
used to select the initial set of peers. 
Note that the KPls have remained 
the same and, therefore, the faculty 
were not consulted again for this 


study. Therefore, no adjustments were 
needed for the updated data set. 


3. Transforming and 
Standardizing Variables 

There was a fair amount of variability 

in enrollment among the initial set of 
institutions. Moreover, the enrollment 
of the target institution was twice 

the size of most of the institutions in 
both data sets. Therefore, some of the 
data elements were standardized to 
mitigate differences due to institutional 
size (Gater, 2003; Huxley, 2009). This 
was accomplished by using the full- 
time equivalent (FTE) for enrollment as 
the divisor. Examples of data elements 
that were standardized by dividing 

by the FTE included the number of 
conferred bachelor’s degrees, number 
of applicants, unduplicated annual 
enrollment, instructional expenses, and 
endowment. 


Full-time and part-time faculty counts 
were combined into one data element. 
In effect, the proportion of full-time 
faculty was calculated by dividing the 
sum of full-time plus part-time faculty 
into the number of full-time faculty. 


4. Determining the Best Set of 
Variables to Use 

Of the 28 variables identified in Step 2, 
three were both output measures and 
among the target institution’s KPIs: (a) 
ratio of conferred bachelor's degrees 
to FTE, (b) 1-year retention rate, and 

(c) 6-year graduation rate. These 
variables were also student centered— 
specifically student success focused— 
and aligned with the target institution's 
mission. To augment the data analysis 
and simplify its interpretation, the 
remaining variables were classified into 
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Table 1. Overall OLS Regression Models for the Three Performance Indicators: Ratio of Conferred Bachelor's Degree to FTE, 
1-Year Retention Rates, and 6-Year Graduation Rates 


Category 
Original Data Set 


Ratio of Conferred Bachelor’s Degrees to FTE 


Variable 


Standardized 


Beta Coefficient 


1-Year Retention Rates 


Admissions 25th Percentile Mathematics SAT .348* 
Faculty Average Faculty Salary —.142 
Enrollment Estimated Fall Enrollment per FTE —.053 
Institutional Characteristics Selectivity —.282** 
Finance Instructional Expenses per FTE -166 


6-Year Graduation Rates 


Admissions 25th Percentile Mathematics SAT 465*** 
Faculty Average Faculty Salary 135 
Enrollment FTE .064 
Institutional Characteristics Selectivity .301*** 
Finance Instructional Expenses per FTE .065 


Updated Data Set 


Ratio of Conferred Bachelor’s Degrees to FTE 


Admissions Percent of Students Receiving Federal Grant Aid —.145** 
Faculty Average Faculty Salary .211** 
Enrollment FTE .090 
Institutional Characteristics Selectivity .274** 

Proportion of Transfer Students —.104** 
Finance Total Price of Attendance .007 

Instructional Expenses per FTE .224*** 
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Admissions Applicants per FTE —.141* 
Faculty Percent of Faculty with Terminal Degree .254** 
Enrollment 12-Month Enrollment per FTE .065 
Institutional Characteristics Selectivity .038 
Finance Total Price of Attendance .393*** 


1-Year Retention Rates 


Admissions 75th Percentile Mathematics SAT .383*** 
Faculty Average Faculty Salary .086 
Percent of Faculty with Terminal Degree .054 
Enrollment FTE .131* 
12-Month Enrollment per FTE —.050 
Institutional Characteristics Selectivity 130 
Finance Total Price of Attendance .089 
Alumni Giving Rate 2.229* 


6-Year Graduation Rates 


Admissions 75th Percentile Mathematics SAT .350*** 
Faculty Average Faculty Salary -.015 
Percent of Faculty with Terminal Degree AZT 
Enrollment FTE .158*** 
12-Month Enrollment per FTE —.040 
Institutional Characteristics Selectivity .132* 
Finance Total Price of Attendance .181** 
Alumni Giving Rate .202*** 


Note: * p <.05, **p <.01, ***p<.001. 


one of the following five groups: (a) 
admissions, (b) faculty, (c) enrollment, 
(d) institutional characteristics, and (e) 
finance. 


As described in our previous research, 
several regression analyses, single- 
step ordinary least square (OLS), were 
used to identify the best variables 

to select a set of peers. In the first 
phase, regression models were 
compiled separately for the five 
variable categories for each of the 
three output measures, a total of 15 
models. Because the analysis was still 
exploratory at this stage, the single- 
step enter method was preferred over 


other models. Distributing the variables 
into five groups allowed the inclusion 
of all variables into the model for that 
category (SPSS, 2012). Informed by 
previous research, the standardized 
beta weights were the determinants of 
what data elements would be used for 
peer selection (Hom, 2008). 


In the second phase, an overall 
regression model for each output 
variable was computed using the 

best predictor(s) from each of the five 
regression models. The best predictor(s) 
had the smallest significance level 
associated with the standardized beta 
coefficient. The standardized beta 


weight's significance level indicates if 

a variable is, in fact, a predictor of the 
output variable (Cohen & Cohen, 1983). 
Although there were some exceptions, 
only one predictor from each category 
was chosen for the three overall 
models. This was deliberate because 
there were high correlations among 
predictors in any given category. In 
addition, the inclusion of only one or 
two predictors from each category 
forced a balance of institutional metrics 
for peer selection. The best predictors 
for each KPI regression model by 
category for the original and updated 
data sets are listed in Table 1. 
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5. Establishing the Best 
Selection Strategy 

Peer institutions are determined by 
having metrics that are close to the 
target institution (McLaughlin et 

al., 2011). This is manifested in the 
computation of a selection index. 
Three selection indices were examined: 
(a) proximity, (b) percentile, and (c) 
normative. 


The calculation of each selection 
index also involves several steps 

but the steps are basically the same 
for each: (a) identifying the most 
relevant parameters, (b) computing 
the numerical difference between the 
comparison and target institutions 
on each of those parameters, (c) 
averaging those differences across 
parameters, and (d) determining range 
cut-scores to delineate a peer from an 
almost-peer. The first step, identifying 
the most relevant parameters, has 
already been decided by the three 
overall OLS models mentioned in 
Step 4. Descriptions of Steps b-c are 
provided for each index below. The 
determination of range cut-scores 

are further described in the “Results” 
section. 


Proximity selection index 

As mentioned, the numeric 
differences between the target and 
each comparison institution were 
computed for each predictor. The mean 
of these differences determines an 
institution’s propinquity to the target 
institution. For the proximity selection 
index, the unit of measurement is the 
standard deviation for each predictor. 
This is depicted in Figure 1, with the 
assumption for this illustration that the 
underlying data distribution for each 
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Figure 1. Selection Index Numeric Assignments for Differences Between Target 
College and Each Institution in the Initial Data Sets 


Numeric assignments for index 
scores indicated in the BLUE boxes 


Proximity -20 -lo -Ao 
Percentile 0 25° 25/2 
Normative Z.. + Zn Zens 


predictor is normally distributed. For 
each predictor, a proximity index score 
of 1 was assigned to the comparison 
institution that was between one-half 
and one standard deviation of the target 
institution's metric, a score of 2 was 
given if the comparison institution was 
within one-half a standard deviation. 
Equally weighted, the average of the 
proximity index scores derives the 
proximity selection index. The two 
equations that compose the proximity 
selection index calculation are shown 

in Appendix B. An example on how to 
calculate the proximity selection index is 
provided in Appendix C. 


Percentile selection index 

For the percentile selection index, 
differences between the target and 
each comparison institution were 
determined for each predictor as it 


0 +%o +lo +20 
O 7"/2 75% 100 
0 Z as Z Le >23 


was for the proximity selection index. 
Moreover, the logic is the same and 

is shown in Figure 1. However, the 
boundaries for each percentile index 
score is determined by the first and 
third quartile cut-scores, and not by the 
data distribution’s standard deviation 
as it was for the proximity selection 
index. In effect, the percentile selection 
index ensures an equal number 

of comparison institutions in each 
partition. 


A slight diversion is in order. Normal 
distributions are not assumed and 
skewed variables can still produce 
accurate results (Smith, 2012). Yet, 
extreme values or outliers on the low 
end or high end of the distribution can 
affect or skew the distribution and drag 
the mean away from a true measure 

of central tendency. Outliers on both 


ends might also affect the distribution’s 
kurtosis. Kurtosis refers to the width of 
the peak of the distribution around the 
measure of central tendency (Hembree, 
2013). In turn, this exaggerated 
dispersion could unduly increase 

the standard deviation and, thus, 
stretch the distribution segments. 
Consequently, a disproportional 
number of comparison institutions 
would receive larger index scores than 
they deserve because they would 

be more likely to fall in a subdivision 
closer to the mean. This might not be a 
problem per se, but could compromise 
the ability of the selection index to 
distinguish a peer from a non-peer. 


On the other hand, the percentile 
selection index distribution is 
partitioned with an equal number of 


comparison institutions in each section. 


Unlike the proximity selection index, 
outliers are less likely to affect the 
percentile selection index because the 
percentile selection index relies on the 
median as the center of the distribution 
and not a potentially displaced mean. 
Therefore, the percentile selection 
index could be advantageous to the 
proximity selection index, especially for 
skewed data distributions. 


For each predictor, a percentile 

index score of 1 was assigned to the 
comparison institution that was within 
25 percentile points of the target 
institution metric, and a score of 2 was 
given if the comparison institution 
was within 12.5 percentile points of 
the target institution. This is a smaller 
partition than the proximity selection 
index, given a percentile index score 
greater than 0 is awarded if the 
comparison institution is within 50 


percentile points or half the percentile 
selection index distribution versus 
approximately 68% of the proximity 
selection index distribution. Equally 
weighted, the average of the percentile 
index scores derive the percentile 
selection index. The two equations 
used for computing the percentile 
selection index are shown in Appendix 
B. An example of how to calculate a 
percentile selection index is provided 
in Appendix C. 


Normative selection index 

Before the boundaries for each 
normative selection index were 
established, values for each predictor 
were converted to z-scores. Each 
predictor was standardized with the 
resulting distribution having a mean 
of 0 and standard deviation of 1 (SPSS, 
2012). That said, the standard normal 
distributions were derived from 

using the original distribution’s mean 
and standard deviation. Therefore, 
effects of the outliers and resulting 
asymmetrical distributions were not 
completely eradicated. However, 

the advantage of these transformed 
distributions is the fact that the new 
distributions were symmetrical. In 
essence, the normative selection index 
is a hybrid of both the proximity and 
percentile selection indices. As with the 
proximity selection index, the mean 
and standard deviation determine 
distance or probability. However, as 
with the percentile selection index, the 
use of the standard normal distribution, 
ensures that the distribution is 
sectioned into equal parts. 


Another benefit of transforming the 
original distribution to the standard 
normal distribution is that the cut- 


points are easier to compute and 
conceptualize. As mentioned, the curve 
created by the z-scores represented by 
the x-axis and resulting probabilities 
plotted on the y-axis, in a standard 
normal distribution is symmetrical 
(Weiss, 2015). The difference in the 
proportion of the total area under the 
curve that is to the right of the z-score 
between the comparison institution 
and target institution was used to 
determine distance from the target 
institution. 


For each predictor, a normative 

index score of 1 was assigned to 

a comparison institution that was 
within one-fourth the distance of the 
total standard normal distribution’s 
area from the target institution. As 
with the percentile selection index, a 
score of 2 was given if the comparison 
institution was within one-eighth of 
the area or distance from the target 
institution's probability corresponding 
to the z-score. Equally weighted, the 
average of the normative index scores 
derives the normative selection index. 
The equations used to compute the 
normative selection index are shown 
in Appendix B. An example on how to 
calculate a normative selection index is 
provided in Appendix C. 


RESULTS 


For the original data set, there were 

58 peers and 47 almost-peers across 
the three peer selection indices. There 
were fewer peers in the updated 

data set, 34. There were 55 almost- 
peers. Across data sets, the normative 
selection index in the original data set 
produced the largest number of peers, 
51. The percentile selection index in the 
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Table 2. Index Score Peer and Almost-Peer Classifications for the Three Selection Indices 


AY-1(-Yeud(odamlale(=).4 


Peer 


N* Percent** 


Original Data Set 


Almost-Peer 


N* Percent** 


Proximity 813 65.6% 426 34.4% 
Percentile 638 53.8% 547 46.2% 
Normative 756 60.7% 487 39.3% 


Uy oye F-\x-Ye ml DY-1e-Mesl-14 


Proximity 750 60.1% 498 39.9% 
Percentile 595 64.5% 327 35.5% 
Normative 606 60.2% 400 39.8% 


Note: * Count of index scores for each predictor for each peer and almost-peer. 
** Percent of index scores that were 1 (Almost-Peer) or 2 (Peer). 


updated data set produced the fewest 
number of peers, 26, just slightly more 
than half the size of the largest set of 
peers or set of almost-peers. 


Selection Index Ranges 
Proximity selection index 

For the original data, the range of the 
resulting proximity selection index 

was 1.33 to 1.78 for the peers and 
almost-peers. The updated data set 
posted a range that was slightly more 
compressed, Proxl Range = 1.44 to 1.78, 
for the peers and almost-peers. The 
cutoffs for the peer set was the 95th 
percentile, while the almost-peers were 
institutions between the 90th and 95th 
percentiles. 


The set of proximity peers and 
proximity almost-peers changed 
between the original data set and the 
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updated data set. In part this was due 
to the smaller set of initial peers in 
2016 compared to 2013 (N = 285,N= 
232, respectively). The smaller number 
of initial peers in the updated data set 
was the result of several circumstances. 
For 46 of the original data initial set 

of institutions, the Basic Carnegie 
classification level changed in 2015 

to a master’s level. The enrollment 

of six of these original data set initial 
institutions dropped below 1,000, and 
one institution closed. 


Examining the individual proximity 
index scores for each predictor in the 
original data set, the proximity index 
scores were more likely to classify a 
comparison institution as a peer than 
an almost-peer (65.6%), although the 
number of peers and almost-peers 
were the same. This is seen in Table 2. 


The updated data set was similar in that 
60.1% of the proximity index scores 
categorized a comparison institution 

as a peer although peers make up only 
two-fifths (42.3%) of both sets (11 vs. 
15, respectively). 


Percentile selection index 

For the original data set, the percentile 
selection index range used to 
determine the peer institutions and 
almost-peer institutions was the same 
as the proximity selection index for the 
updated data set (Percl Range = 1.44 
to 1.78) but more compressed than 
the percentile selection index for the 
updated data set (Percl Range = 1.11 
to 1.56). For comparative purposes, the 
same cutoffs used for the proximity 
selection index were also applied to 
the percentile selection index, 95th 
percentile or higher for peers and 


Figure 2. Selection Index Distributions 


00 50 1.00 1.50 
Proxindex 


00 50 1.00 1.50 
Proxindex 


between the 90th and 95th percentiles 
for almost-peers. 


As seen in Table 2, the original data 

set, the percentile selection index 
methodology had a more equitable 
split between the two peer groups with 
almost 54% (53.8%) of the percentile 
index scores classifying comparison 
institution as peers. However, the peer 
set is more than twice the size of the 
almost-peer set (21 vs. 9, respectively) 
and, therefore, the percentile index 
scores do not follow the individual 
percentile index score classification 
proportions. For the updated data 

set, about one-third (35.5%) of the 
percentile index scores categorized a 
comparison institution as an almost- 
peer, but the number of percentile 
selection index peers and almost-peers 
is similar (12 vs. 14, respectively). 


Percindex 


UPDATED DATA SET 
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Normative selection index 

For the original data set, the normative 
selection index range (Norml = 1.22 

to 1.89) was larger than the other 
selection indices. Therefore, the range 
for the updated data set was more 
compressed (Norml = 1.38 to 1.75) than 
the range for the normative selection 
index in the original data set. Again, 
the same cutoffs used for the other two 
selection indices were also applied to 
the normative selection index: 95th 
percentile or higher and between the 
90th and 95th percentiles for peers and 
almost-peers, respectively. 


In the original data set, the proportion 
of individual normative index scores 
that classified a comparison institution 
as a peer (60.7%) is the inverse of the 
actual proportion of peers (36.7%) to 
almost-peers. For the updated data 


oo 50 1.00 1.50 2.00 
Normindex 


set, the proportion of index scores that 
classified a comparison institution as a 
peer (60.2%) is more analogous to the 
actual proportion of peers and almost- 
peers, with more than half (55.6%) of 
the institutions at or above the 95th 
percentile. 


Selection index distributions 

An examination of each selection 
index distribution is shown in Figure 2. 
As seen, the distributions are shaped 
differently from what was expected. 
For example, in the original data set 
the proximity selection index should 
be very susceptible to outliers, but in 
fact it was more normally distributed 
than the percentile selection index 
that had a more pronounced right 
skewness. The probabilities associated 
with the percentile selection index 
were moderately uniform, yet 
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Table 3. Skewness and Kurtosis for the Predictors and Each Selection Index 


Selection Index Skewness Kurtosis 
25th Percentile Mathematics SAT .56 —.15 
Percent of Students Receiving 83 .28 
Federal Grant Aid 

Average Faculty Salary 74 99 
FTE 95 72 
Total Price of Attendance -.01 —.22 
Instructional Expenses Per FTE 1.79 4.10 
Alumni Giving Rate .70 37 
Proximity Selection Index .00 -.51 
Percentile Selection Index —.03 —1.12 
Normative Selection Index 02 —.69 


Updated Data Set 


Applicants per FTE .80 .50 
75th Percentile Mathematics SAT -1.44 —.33 
Percent of Faculty with Terminal Degree -1.44 1.97 
Average Faculty Salary 54 38 
FTE 1.08 1.26 
12-Month Enrollment Per FTE 4.01 21.06 
Total Price of Attendance -.11 —.79 
Alumni Giving Rate —.82 63 
Proximity Selection Index -.31 -.71 
Percentile Selection Index —.01 —.57 
Normative Selection Index .06 -.71 


multimodal or with more than one 
peak. For the updated data set, the 
proximity selection index distribution 
was more left skewed than both the 
percentile and normative selection 
index distributions and all distributions 
generated by the original data set. 
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All but the normative selection index 
distribution is misshapen. Again, the 
percentile index distribution appears to 
be multimodal. 


To further investigate these 
incongruities, the skewness and 


kurtosis for each selection index were 
also computed. Results are shown 

in Table 3. In brief, the differences 

in asymmetry of the selection index 
distributions affect peer selection. 


Paradoxically, only the proximity 
selection index distribution for the 
original data set was not left skewed. 
This is shown in Table 3 and Figure 

2. Although slight, the percentile 
selection index was the most skewed 
(SE = -.03) of the original data set 
selection indices. For the updated 

data set, the proximity selection index 
posted the largest skew (SE = -.31) and 
the percentile selection index had the 
smallest skew (SE = -.01). Overall, the 
distributions gleaned from the updated 
data set seem to be more normally 
shaped than the original data set 
distributions. 


Delving deeper into the data, it was 
discovered that the predictors used 
in the selection indices were also 
skewed. Skewness and kurtosis for 
the continuously scaled predictors 
are also shown in Table 3. Except for 
the Total Price of Attendance (SE = 
-.01) predictor, all were positively 

or right skewed in the original data 
set. The Instructional Expenses Per 
FTE predictor was the most skewed 
(SE = 1.79). For the updated data set, 
one-half (4) of the predictors were 
left skewed and one-half were right 
skewed. The 12-Month Enrollment 
Per FTE predictor (SE = 4.01) was the 
most skewed. Yet, even with the more 
pronounced skewness of the predictors 
in the updated data set compared to 
the original data set, the equitable 
proportion of left skew to right skew 
predictor distributions in the updated 


data set seemed to balance all the 
selection index distributions. 


Examining the selection index 
distributions’ kurtosis can also be 
informative. As seen in Table 3, all 

the selection index distributions for 
both the original and updated data 
sets had negative kurtoses. Negative 
kurtosis is associated with distributions 
with a flatter distribution compared 
to a normal distribution. A normal 
distribution would have a kurtosis of 
zero (DeCarlo, 1997). Not surprisingly, 
the percentile selection indices had 
the most negative kurtosis, indicating 
that it is less peaked than the other 
distributions. This is decipherable in 
Figure 2. In this regard, the selection 
index is working as intended. On the 
other hand, for the updated data set 
the kurtosis was similar across selection 
indices. The percentile selection index 
was the most peaked, albeit negative. 


Selection index combinations 
Comparison institutions were seldom 
chosen for membership in more 

than one selection index peer group. 
This is shown in Table 4. The original 
data set peer groups have the most 
overlap with one-third (33.3%) of 

the comparison institutions either 

a proximity or normative selection 
index peer.. This might be because 

the distribution for the proximity peer 
selection index is normally shaped and, 
therefore, the percentile and normative 
peer selection index transformations 
did not make much of a difference. For 
the updated data set, no comparison 
institution was a member of all three 
peer selection indices and only four 
comparison institutions (5.9% each for 


Table 4. Peer Overlap Across Peer Selection Indices 


Selection Index/ices 


Percent Overlap 


Original Data Set 


Proximity/ Percentile 


Almost-Peer 


Proximity/ Normative 


33.3% 21.3% 


Percentile/ Normative 


23.8% 6.4% 


All 3 


Updated Data Set 


Proximity/ Percentile 


Proximity/ Normative 


2.9% 20.0% 


Percentile/ Normative 


0.0% 0.0% 


proximity/ percentile and percentile/ 
normative peer selection index 
combinations) were chosen for two 
peer selection index groups. Again, the 
peer selection index distributions— 

or, more precisely, the differences 
among the distributions—could have 
contributed to the uniqueness of each 
peer selection index membership. 

The proximity selection index is left 
skewed, the percentile selection index 
is multimodal, and the normative 
selection index is the most symmetric 
but slightly right skewed. Note that, 
unlike the original data set, overlap 
among peers was more prevalent for 
the three almost-peer selection index 
groups compared to the peer selection 
index groups. 


Feasibly, symmetry could be achieved 
by combining selection indices, as 
was the case for the updated data set. 
Comparison institutions that were 

(a) only a normative selection index 
peer (NORMATIVE ONLY), (b) both a 
percentile and a normative selection 
index peer (BOTH), or (c) neither a 
percentile nor a normal selection 
index peer (NEITHER), were further 
investigated. As a starting point, the 
difference or distance between the 
average of each of these selection 
index peer sets and the target 
institution were examined for each 
continuously scaled predictor. This is 
seen in Figure 3. For the original data 
set, target institution was closest to the 
means derived from BOTH institutions 
for almost half (42.9%) of the seven 
predictors. For the updated data set, 
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Figure 3. Target Institution Comparisons to the Normative Selection Index 
Peers Only, Both Normative and Percentile Selection Index Peers, and Neither a 
Normative or Percentile Selection Index Peer 


Original Data Set 
NORMATIVE BOTH NEITHER 
perrerceniewanenatessaT |X [| 


Instructional Expenses Per FTE - 
Alumni Giving Rate 


om ee 


Updated Data Set 
NORMATIVE BOTH NEITHER 
eons TX] CdS 


12 Month Enrollment/ FTE 


Alumni Giving Rate 


Sa ee Ee 


Both Data Sets 
NORMATIVE BOTH NEITHER 
TOTAL a a 


PERCENT 33.3% 53.3% 13.3% 


NORMATIVE: Normative Selection Index Peers Only 
BOTH: Peers that are both Percentile and Normative Selection Index Peers 


NEITHER: Peers that are neither Percentile nor Normative Selection Index Peers 
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BOTH institutions posted predictor 
means closest to the target institution 
for over half (62.5%) of the eight 
predictors. Combined across data sets, 
the target institution was closer to the 
institutions that were BOTH peers more 
frequently (53.3%) than the other two 
groups. Next was the NORMATIVE ONLY 
institutions, with one-third (33.3%) of 
the predictor means being nearest to 
the target institution compared to the 
other two groups. The NEITHER peer 
institutions fared the worst, with the 
distance between the target institution 
and the peer institution being the 
closest for only two predictors (13.3%) 
across data sets. In sum, institutions 
that are both percentile and normative 
selection index institutions tended to 
be the nearest to the target institution 
compared to the normative selection 
index-only institutions or those 
institutions that were in neither the 
percentile nor normative selection 
index peer groups. 


A closer examination of the target 
institution's position on each 
continuously scaled predictor’s 
distribution corroborates these 
findings. In Figure 3 green indicates 
the position of the target institution at 
the high end (right) of that predictor’s 
distribution, yellow indicates the 
target institution in the middle of 

the predictor’s distribution, and red 
indicates the target institution at the 
low end (left) of the distribution. As 
seen, there was no clear pattern. That is, 
the target institution’s position on the 
distribution did not seem to influence 
peer selection index membership. 
This might be good news, in that the 
selection indices were somewhat 


unaffected by the target institution 
performance compared to other 
institutions. 


CONCLUSION 


This study investigated the use of 
three peer selection indices: proximity, 
percentile, and normative. These 
selection indices were applied toa 
predetermined set of institutions using 
institutional characteristics based on 
constituent feedback and institutional 
priorities. To select a set of peers that 
was well-informed and aligned with 
those priorities, the following steps 
were executed: (a) determination of 
what data to use, (b) data element 
standardization, (c) regression 
modeling to identify the predictors 
that were best correlated with key 
institutional attributes, (d) computation 
of index scores and corresponding 
selection indices, and (e) ascertaining 
the appropriateness of the selection 
indices. The last step was accomplished 
by comparing peer sets that were 
identified for each selection index 

to each other as well as considering 
the impact of the distributions of the 
predictors that make up each selection 
index. 


As mentioned, the crux of the paper 
was to describe each selection index 
and to explore the differences among 
the three selection indices’ peers. This 
research is innovative in that this was 
the first time that two of the selection 
indices, percentile and normative, were 
formally introduced and investigated. 
Moreover, the three selection indices 
were investigated simultaneously. As 
with our previous research, no selection 
index is endorsed outright but rather 


the plausibility and limitations of each 
was discussed. That said, selection 
index methodology holds promise as 
a robust and legitimate peer selection 
tool. 


Because of the number of institutions 
in the initial data sets (N = 285 and N= 
232 for the original and updated data 
sets, respectively), the 95th percentile 
of the selection index was established 
as the cutoff for choosing peers. 
Another set of almost-peers was also 
identified from institutions that were 
between the 90th and 95th percentiles. 
The two-tiered system to classify 

the immediacy of the institutions 

to the target institution is practical 
because of the relatively small range 
of index scores for all three selection 
indices. In turn, there might not be 
any meaningful differences regarding 
nearness to the target institution 
between institutions in the 90th to 
95th percentile range and those in the 
95th percentile to maximum range. 


The results are not conclusive, but 
nonetheless indicate that using 
selection index composites—in 
particular a combination of the 
percentile and normative selection 
indices—can be useful. Although not 
a factor for the data sets used in this 
research, being mindful of the target 
institution's distribution position could 
be important and warrants further 
investigation. 


Comparisons between the original data 
set and the updated data set reinforce 
the importance of regularly verifying 
an institution's list of peers. The peer 
institutions that were chosen changed 
over time, regardless of the selection 


index used. In fact, less than one-half 
(44.7%) of the proximity selection 
index peers or almost-peers identified 
in the updated data set were part 

of the original data set of proximity 
selection index peer or almost-peer 
list. The percentile selection index was 
somewhat less stable across data sets. 
Only one-third (33.3%) of the original 
data set percentile selection index peer 
or almost-peer institutions made the 
updated data set percentile selection 
index peer or almost-peer list. Likewise, 
only one-third (33.3%) of the normative 
selection index peer or almost-peers 

in the original data set were also peers 
in the updated data set. That said, the 
original data set of normative selection 
index almost-peers was very large 
compared to the other normative 
selection index peer sets, essentially 
ensuring some correspondence. 


Admittedly, the Carnegie Classifications 
were modified between the extraction 
of the original and updated data sets 
and those modifications affected the 
selection of the initial set of peers and, 
ultimately, the final selection of peers. It 
is expected that Carnegie Classification 
will be updated more frequently and, 
therefore, the time of extraction of the 
two data sets used in this study was 
apropos. The results of this study can 
be taken as a warning that peer lists 
can become outdated and unsuitable. 
As this research demonstrates, it is 
reasonable to expect that institution 
characteristics and priorities change 
over time. 


Finally, peer list differences among 
the selection indices demonstrate 
the importance of due diligence 

before, during, and arguably even 
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after the selection process. Feedback 
from faculty and staff are key to this 
thoroughness. Beforehand, engaging 
constituent input not only helps 

to identify institutional priorities 

but, afterward, also reinforces their 
importance. Additionally, participation 
of constituents increases acceptance 
and use of the chosen set of peers. 


Examining the set of institutions 
gleaned by each selection index 
affords both a comparison of the 
appropriateness of each institution 

as a peer and the set of institutions as 
a reasonable peer group (D’Allegro & 
Zhou, 2013). To that end, the choice of 
initial set of institutions is crucial. These 
institutions should be approximate to 
the target institution by proxy of both 
institutional characteristics and the 
predictors that will ultimately be used 
to choose a set of peers. 


RECOMMENDATIONS 


This study validates that peer 
selection based on a multistaged 
approach is necessary but not 
sufficient. Careful vetting of the 
appropriateness of the actual 
statistical steps and methodology are 
needed. As an example, several OLS 
regression models were generated 

to determine the best predictors of 
institutional quality, the mainstay of 
the target's priorities. However, other 
methodologies could be employed, 
including discriminant analysis, factor 
analysis, and variable match (Anderes, 
1999). 


Preliminary scrutiny of the variables to 
choose peers should be undertaken. 
To ensure the best mix of institutional 
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characteristics to choose peers, 

this research engaged a two-stage 
regression modeling process. In the 
first stage, the best predictors were 
chosen from five different institutional 
characteristic categories. The second 
stage confirmed the correlation of the 
predictors to three institutional quality 
measures. Additionally, the location 

of the target institution on each 
potential predictor distribution and 
other anomalies should be identified 
and considered a priori to the actual 
determination of peers. 


The examination of the selection 
indices is also in order. As the 
comparison of the distributions for 
each selection index revealed, resulting 
non-normal distributions had a 
profound impact on the selection of 
peers. 


Both the type of institution as well 

as the purpose of the peer selection 

are key in determining the most 
appropriate information to collect 
(Shin, 2009). The use of historical 
information and data trends are posited 
as options but might not be fitting. As 
was the case in this study, historical 
information gleaned a different set of 
peers than more-current data. 


Following the logic of the use of a two- 
tier taxonomy, two sets of peers were 
identified: peers and almost-peers. This 
affords the flexibility of choosing peers 
for different purposes and audiences. 
In addition, it somewhat mutes the 
imperfections of the peer selection 
methodologies. 


The purpose of the study was to 
provide reasonable peer selection 


options. As stated, peer comparisons 
have many applications, such as 
determining quality, benchmarking 
salaries, evaluating programs, 
informing policy, and setting strategic 
direction. Coupled with the wide 
variety of institutional types and 
missions and the inconclusiveness of 
the results, no single selection index 
can be upheld to be better than the 
other selection indices. Accordingly, 
care should be taken to determine the 
best selection index or combination 
of selection indices. As for the latter, 
selection index combinations should 
be further investigated. A set of 
institutions determined to be a peer 
by two or more selection indices might 
prove to be more trustworthy and 
steadfast than the selection of peers 
from only one selection index. This 
seemed to be the case in this study, 

in which there was less distance from 
the target institution for most of the 
predictors for the combined selection 
index peers compared to the normative 
selection index-only peers, or, for 
comparison, institutions not selected 
by either the percentile or normative 
selection indices. 


As of this study, there are few 
publications on peer selection 
methodologies. Evidence that is more 
conclusive is needed about peer 
selection models and the effect that 
target institution type might have 

on those models. As mentioned, 

the impact of peer comparisons on 
institutional quality and improvement 
has not been studied. Evaluation that 
invokes the use of peers seems to be 
in vogue but the question remains: Are 
peer comparisons or benchmarking 
superior to other types of comparative 


assessments or non-comparative 
evaluation? Further research should be 
able to address. 
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Appendix A. Data Elements Used for Peer and Aspirant Selection: Time Frame, Indicator Type, and Source 


WET] 0) (1 Time Frame Tate [rer=1 Xo) amy AY] of -1 Indicator Source 
Admit Yield 2011-2012, 2014-2015 Admissions IPEDS 
Number of Applicants, Total 2011-2012, 2014-2015 Admissions IPEDS 
Percent of Applicants Admitted 2011-2012, 2014-2015 Admissions IPEDS 
SAT Critical Reading 25th Percentile 2010-2011, 2014-2015 Admissions IPEDS 
Score 

SAT Critical Reading 75th Percentile 2010-2011, 2014-2015 Admissions IPEDS 
Score 

SAT Math 25th Percentile Score 2010-2011, 2014-2015 Admissions IPEDS 
SAT Math 75th Percentile Score 2010-2011, 2014-2015 Admissions IPEDS 
Percent of Full-Time Undergraduates 2010-2011, 2013-2014 Admissions IPEDS 
Receiving Federal Grant Aid 

Average Salary Equated to 9-Month 2011-2012, 2014-2015 Faculty IPEDS 


Contracts of Full-Time Instructional Staff: 
All Ranks 


Full-Time Primary Instruction Head Fall 2011, Fall 2015 Faculty IPEDS 

Count 

Part-Time Primary Instruction Head Fall 2011, Fall 2015 Faculty IPEDS 

Count 

Percentage of Faculty Holding Terminal | 2011-2012, 2015-2016 Faculty U.S. News & World 

Degrees Report 

Estimated Fall Enrollment Fall 2010, Fall 2015 Enrollment IPEDS 

Full-Time Equivalent (FTE) Fall 2010, Fall 2015 Enrollment IPEDS 

Total Enrollment, Unduplicated 2010-2011, 2014-2015 Enrollment IPEDS 

Percentage of Classes Enrolling Fewer | 2011-2012, 2015-2016 Enrollment U.S. News & World 

Than 20 Students Report 

Carnegie Classification: Basic (Arts & 2010, 2015 Institutional Characteristic IPEDS 

Sciences or Diverse Fields) 

Carnegie Classification: Enrollment Size | 2010, 2015 Institutional Characteristic IPEDS 

& Setting 

Carnegie Classification: Undergraduate | 2010, 2015 Institutional Characteristic IPEDS 

Profile (Transfer & Full-Time Propor- 

tions) 

Geographic Region 2011-2012, 2014-2015 Institutional Characteristic IPEDS 

Religious Affiliation 2011-2012, 2014-2015 Institutional Characteristic IPEDS 

Endowment (FASB) 2009-2010, 2013-2014 Financial IPEDS 

Instructional Expenses Per FTE (FASB) | 2009-2010, 2013-2014 Financial IPEDS 

Tuition Total Price for In-District Students | 2011-2012, 2014-2015 Financial IPEDS 

Living on Campus 

Alumni Giving Rate 2011-2012, 2015-2016 Financial U.S. News & World 
Report 

Bachelor’s Degrees Conferred 2010-2011, 2014-2015 Student Success IPEDS 

Graduation Rates, Total Cohort (6 Years) | As of Aug. 31, 2010, As of Aug. | Student Success IPEDS 

31, 2014 
Retention Rates, Total Cohort (1 Year) Fall 2010, Fall 2014 Student Success IPEDS 
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Appendix B. Equations Used to Compute Each Selection Index 


PROXIMITY SELECTION INDEX EQUATIONS 


ProxSvar1= (Tlvarx — Clvarx)/ SDvarx 
varx €{1,...x} 


Proxinstitution= average (ProxSvar1...ProxSvarx) 
institution e€{1,...n} 


Where: 
ProxS = Proximity Score 
Proxl = Proximity Selection Index 
TI= Target Institution 
Cl= Comparison Institution 
Var1-Varx= Predictors 


0 assigned to ProxS when: ProxS < -1 or ProxS > 1 
1 assigned to ProxS when: -1< ProxS < -.5 or .5 < ProxS < 1 
2 assigned to ProxS when: -.5 < ProxS < .5 


PERCENTILE SELECTION INDEX EQUATIONS 


PercSvar1= (Tlvarx — Clvarx) 
varx €{1,...x} 


Perclnstitution= average (PercSvar1...PercSvarx) 
institution e{1,...n} 


Where: 
PercS = Percentile Score 
Percl = Percentile Selection Index 
TI= Target Institution 
Cl= Comparison Institution 
Var1-Varx= Predictors 


0 assigned to PercS when: PercS < -.25 or PercS > .25 
1 assigned to PercS when: -.25 < PercS < -.125 or .125 < PercS < .25 
2 assigned to PercS when: -.125 < PercS < .125 


NORMATIVE SELECTION INDEX EQUATIONS 


NormSvar1= (Tlvarx — Clvarx) 
varx e{1,...x} 


Norminstitution= average (NormSvar1...NormSvarx) 
institution €{1,...n} 


Where: 
NormS = Normative Score 
Norml = Normative Selection Index 
TI= Target Institution 
Cl= Comparison Institution 
Var1-Varx= Predictors 


0 assigned to NormS when: NormS < -.25 or NormS > .25 
1 assigned to NormS when: -.25< NormS < -.125 or .125 < NormS < .25 
2 assigned to NormS when: -.125 < NormS < .125 
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Appendix C. Examples on How to Calculate Each Selection Index 


PROXIMITY SELECTION INDEX 
Target Comparison 
Notation Institution Institution 
Var, Predictors Value Value Deviation 


Five, | FTL e109 | 1400 | 200 | rarora0nya00 | 
ves [emt Satals —eoe[s sans tse nara sons 
Var. Faculty Salary} $ 45,000 | $ 50,000 45,000-50,000)/15,000 -.5 and ProxS < .5 
ve | Tisiadls aools vals co leona siaase 
Var. Attendance] $ 42,000 | $ 37,500 | $ 6,000 | (42,000-37,500)/6,000 -.5 and ProxS <-1 


AVE of ProxS,, ProxS,, ProxS, 1.00 


x 
2 
= 
E 
x 
& 
a 


* O Assigned to ProxS when: ProxS <-1 or ProxS > 1 
1 Assigned to ProxS when: -1< ProxS <-.5 or .5 < ProxS < 1 
2 Assigned to ProxS when: -.5 < ProxS <.5 


PERCENTILE SELECTION INDEX 


Target Comparison 
Notation Institution Institution 
Var Predictors Value Value 
| 1810 | 1400 


| ver, | Fecuny Salary] $ 45,000 
Var Faculty Salary} $ 45,000 | $ 50,000 
Var. Attendance] $ 42,000 | $ 37,500 


Percentile Index 


Target Comparison 
Institution Institution 
Var. Predictors Value Value 
| va, | FT| 55.00% | 40.00% | 56.40 
ves Iressa] wun | om [me | on [2 [Pema 
Var. Faculty Sala 50.00% 62.00% and PercS < .125 


ree Pe ee 
and PercC < .25 
Var. Attendance 62.00% 32.00% 


* 0 Assigned to PercS when: PercS <-.25 or PercS > .25 
1 Assigned to PercS when: -.25< PercS <-.125 or .125 < PercS < .25 
2 Assigned to PercS when: -.125 < PercS < .125 


1.00 


Ave of PercS,, PercS,, PercS, 
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Appendix C. Examples on How to Calculate Each Selection Index 


NORMATIVE SELECTION INDEX 


Target Comparison 
Notation Institution Institution 
Var Predictors Value Value 


| Var, | FTE] 4810] 1,400 


| vee, recumy Samy] $ —45,000/$ 50.000] 
Var Faculty Salary| $ 45,000 | $ 50,000 
Pe Be Pe 

Var Attendance] $ 42,000 | $ 37,500 


Converted to the Area Corresponding to the Standardized Normal Scores* 


Target Comparison 
Notation Institution Institution 
Var, Predictors Value Value 
Because Norms > 
ee ee ee _ 
Var, 35-45 NormS < 125 


Normative Index 


Average Because oe >-.25 
Faculty Salary 0.88 wean nee : 
Total Cost of 
ey PP OS 


Ave of NormS,, NormS,, NormS, 


* To compute the area: 
1. Calculate the z-score= (Value, - Mean,..)/StdDev,,,,, 
2. Find the area under the curve to the right of the z-score 
(NORMS.'NV function in Excel) 


** 0 Assigned to NormS when: NormS < -.25 of NormS > .25 
1 Assigned to NormS when: -.25< NormS <-.125 of 125 < NormS <.25 
2 Assigned to NormS when: -.125 < NormS < .125 


SUMMER 2017 VOLUME | PAGE 53 


Thank You! 


AIR expresses sincere appreciation for the members who serve as peer reviewers. Authors who submit 
materials to AIR Publications receive feedback from AIR members. The following individuals reviewed 
manuscripts submitted to this volume of Professional Files. 


Matt Anson Nora Galambos Pa Thao 
Dave Becher Jennifer Kobrin Benita Thompson 
Anne Marie Brady Ethan Kolek Ying Vuthipadadon 
René Cintrén Teresa Isbell Annie Weber 
Amanda Clark Jessica Lillegaard Mike Williford 
Linda Clark Sun No 
Liying Cui Gee Lockhart Sigman 


About AIR Professional File 


Professional File volumes are journal-length publications grounded in relevant literature that synthesize 
current issues, present new processes or models, or share practical applications related to institutional 
research. All submissions are peer-reviewed. For more information about AIR Publications, including 
Professional File and instructions for authors, visit www.airweb.org/publications. 


Association for Institutional Research 
Christine M. Keller, Executive Director and CEO 


www.airweb.org 


ISSN 2155-7535 


AIR provides higher education professionals with learning opportunities that better equip them 
to meet the needs of data-informed decision making at their institutions. Whether online or 
face-to-face, in the office or at our annual conference, AIR’s professional development 
opportunities provide outstanding return on investment for those working in IR and related 
fields, and the institutions they represent. 


A Holistic Approach to IR 
This course provides a foundation for participants to meet and navigate the ever-growing 
demands for data and information in the current higher education landscape. 


IPEDS Keyholder Essentials: A Beginner's Guide 
Created for data providers with less than 9 months of experience as keyholders, this 
course covers basic concepts and definitions. 


IPEDS Keyholder Efficiencies: Reducing the Reporting Burden 
Created for data providers with 10-24 months of experience, this course is designed to 
enhance your knowledge as an IPEDS Keyholder. 


Data and Decisions Academy® 
The Data and Decisions Academy is a series of online, self-paced courses designed for 
community college IR professionals. 


AIR Forum 

Learn, connect, and share at the largest gathering of IR professionals in the world, and 
take advantage of recorded sessions, presentation slides, handouts, and scholarly papers 
available after the conference. 


it 
AIR ASSOCIATION FOR INSTITUTIONAL RESEARCH 
Data and Decisions for Higher Education 


LEARN MORE AT WWW.AIRWEB.ORG 


